Conversation
|
@deruyter92 can you fix conflicts? |
There was a problem hiding this comment.
Pull request overview
This PR improves the handling of cases where no detections are found and refactors how multi-animal mode is determined. The changes skip pose estimation entirely when there are no detections and return an all-zeros pose, while also inferring the single vs. multi-animal mode from the model's metadata configuration rather than requiring it as a parameter.
Changes:
- Returns all-zeros pose when no supra-threshold detections are found, skipping the pose estimation step
- Infers single/multi-animal mode from model metadata, deprecating the
single_animalparameter - Adds
n_individualsandn_bodypartsattributes extracted from model configuration
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@deruyter92 can you check suggestions and fix conflicts, thanks! |
deruyter92
force-pushed
the
jaap/fix_no_detections_case
branch
from
d9f2d5c to
dcaa476
Compare
- add fields for n_individuals and n_bodyparts read from the pytorch model config. - add option to infer single_animal mode from n_individuals in model config. This commit does not change default behaviour. Only when passing single_animal = None explicitly to PyTorchRunner, the single_animal mode will be inferred from the model config.
deruyter92
force-pushed
the
jaap/fix_no_detections_case
branch
from
dcaa476 to
6d587c1
Compare
When the detector does not detect any crops (with a supra-threshold confidence), no pose-detection is applied and and a zero-vector is returned for the pose-prediction. This solves DeepLabCut#137 and copies the already existing intended behavior of returning zeros for empty pose-predictions in single animals.
- For zero detections, the age is still incremented - `_detections` attrubute is set to None
deruyter92
force-pushed
the
jaap/fix_no_detections_case
branch
from
6d587c1 to
c618a7f
Compare
|
Simplified this PR. Should be ready for review now.
|
|
Tests for this? Seems like getting the model to give you no detections should be straightforward by just feeding it a blank or noise frame, and since this bug caused a crash in a common case, seems worth protecting. Also as a general question (not sure if this problem is specific to some torch-only functionality): i see this behavior added in the torch runner, not the base runner - i am sort of assuming yes, but are the tensorflow models deprecated? If behavior starts to diverge/updates are made to torch but not tf, we might want to stick a deprecation warning on the tf parts. |
|
TF definitely not depreciated -- in active use in my lab :D but the bug is only related to PT models 👍 |
|
Closing this pull request, as the original issue #137 has a different cause, and therefore these changes are not required. |
4 participants
Motivation
For top-down models, the downstream pose processing should not occur when there are no detections, and the output shapes should always remain consistent. Instead of using
offsets_and_scales = [(0, 0), 1], which potentially interrupts/breaks the pipeline, a zero-pose is returned following the earlier implementation for single animals.Changes
get_pose(..)returns a pose with all-zeros. Fixes DeepLabCut-live fails to run exported superanimal_humanbody model from DeepLabCut #137, and follows earlier intended implementation for single animals. Now it should work for multiple detections as well, and the pose-estimation step is entirely skipped when there are no detections.2. Infer multi-animal mode from the metadata configuration, in theload_modelstage, rather than the argumentsingle_animalat initialization. This has the advantage that a PytorchPoseRunner can be initialized with only an exported model path, without defaulting to single animal mode.single_animalparameter from the number of individuals in the PyTorch configuration metadata field. The default behavior remains unchanged (single_animal=True).